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DESCRIPTION 



SOLID-STATE COLOR IMAGE PICKUP APPARATUS 



TECHNICAL FIELD 

The present invention relates to a progressive scan type 
solid-state color image pickup apparatus and^ more particularly, 
to an arrangement of a color separation filter and a matrix 
calculation method for suppressing degradation in resolution of 
a luminance signal, which occurs when data from the color 
separation filter is subjected to matrix calculation. Further, 
the invention relates to a signal processing method for a solid- 
state color image pickup apparatus which obtains high resolution 
by performing interpolation between pixels. In this method, 
when a color-difference signal is composited from chrominance 
signals outputted from a solid-state image pickup device, the 
frequency characteristic of each chrominance signal is adjusted 
for each color of the color-separation filter to reduce a 
frequency component including aliasing which causes pseudo 
chrominance s ignals • 

BACKGROUND ART 

A video signal is usually represented by three primary 
colors of light: red (R), green (G), and blue (B), and it is 
also represented by luminance signals (Y) and two kinds of 




color-difference signals (R-Y^ B-Y) . The three primary colors R, 
G, B usually take a form of an input signal to a monitor for a 
computer, and the luminance and color difference signals take a 
form of a digital part in a device such as a TV. Further, in 
recent years, a video signal from a solid-state color image 
pickup apparatus is used not only for image display but also for 
digital recording or video communication between devices . A 
video signal has a great amount of data, and usually it is 
compressed because of the limited recording capacity or 
:j communication capacity. A format of video signal employed in 
J the video compression process is called 4:2:0 or 4:1:1 format in 
Id which chrominance data is reduced to half as compared with 4:2:2 
pi format which has conventionally been employed, 
fy Hereinafter, the conventional solid-state color image 

□ pickup apparatus will be described with reference to figure 2. 

Figure 2(a) shows the conventional solid-state color image 
pickup apparatus, wherein 1 denotes an optical system for 
forming an image of a subject on the surface of a solid-state 
image pickup device; 2 denotes a solid-state image pickup device 
with a color separation filter for converting the image of the 
subject (optical image) to an image signal (electric signal); 3 
denotes an AD converter for converting the image signal obtained 
by the solid-state image pickup device to a digital image 
signal; and 4 denotes an image signal processing circuit for 
converting the digital image signal to a luminance signal and 
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color-difference signals. Usually, a color separation filter 
having a checkered pattern of complementary colors shown in 
Figure 2(b) comprising magenta (Mg), green (G), cyan (Cy), and 
yellow (Ye) is used for the color separation filter placed on 
the surface of the solid-state image pickup device 2. 

The operation of the solid-state color image pickup 
apparatus so constituted will be described hereinafter. 

In Figure 2, the subject image (optical image) is formed 

l;^; on the solid-state image pickup device by the optical system 1. 

:J The solid-state image pickup device 2 outputs the subject image 
as an image signal which is color-separated by the color 

f"' separation filter. The image signal is converted to a digital 

Lrii signal by the AD converter 3, supplied to the signal processing 
circuit 4, and converted to a luminance signal Y and two kinds 

h: of color-difference signals, whereby a color video signal is 
obtained. The image signal processing circuit generates a 
luminance signal (Y) and a pair of color-difference signals (R-Y, 
B-Y) corresponding to each pixel, from the four pixels of 
complementary colors Mg, G, Cy, and Ye. 



An example of a procedure to generate luminance signals is 
as follows. Luminance signals Y(h,v) are represented by 



Y(0,0) 



Mg ( 0 , 0 ) +G ( 1 , 0 ) +Cy (0,1) +Ye (1,1) 



Y(1,0) 



G(l,0)+Mg(2,0)+Ye(l,l)+Cy(2,l) 



Y(0,1) 



Cy(0,l)+Ye(l,l)+G(0,2)+Mg(l,2) 



Y(l,l) 



Ye(l ,1)+Cy(2,l)+Mg(l,2 )+G(2,2) 
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and thus four luminance signals are generated from the outputs 
of the solid-state image pickup device corresponding to the four 
pixels . 

Likewise^ color-difference signals R-Y (h,v) are 
represented by 

R-Y(0,0) = Mg(0,0)-G(l,0)-Cy(0,l)+Ye(l,l) 
R-Y(0,1) = -G(l,0)+Mg(2,0)+Ye(l,l)-Cy(2,l) 
R-Y(1,0) = -Cy(0,l)+Ye(l,l)-G(0,2)+Mg(l,2) 
R-Y(l,l) = Ye(l,l)-Cy(2,l)+Mg(l,2)-G(2,2) 
and color-difference signals B-Y (h,v) are represented by 
B-Y(0,0) = Mg(0,0)-G(l,0)+Cy(0,l)-Ye(l,l) 
B-Y(0,1) = -G(l,0)+Mg(2,0)-Ye(l,l)+Cy(2,l) 
B-Y(1,0) = Cy(0,l)-Ye(l,l)-G(0,2)+Mg(l,2) 
B-Y(l,l) = -Ye(l,l)+Cy(2,l)+Mg(l,2)-G(2,2) 
The number of the luminance signals Y, color difference 
signals R-Y, and color difference signals B-Y are equal to the 
number of pixels of the solid-state image pickup device, that is, 
these signals take the 4:4:4 format. According to a device to 
which these signals are applied, conversion to 4:2:2, 4:2:0, or 
4:1:1 is carried out. 

Moreover, since pseudo chrominance is generated at edges 
of luminance, edge judgement is performed according to edge 
signals from the composed luminance signals, and the gains of 
the color-difference signals corresponding to pixels decided as 
being edges are reduced to suppress the pseudo chrominance. 



• 

However, in the conventional solid-state color image 
pickup apparatus, no consideration is given to image compression, 
and it is premised that the output signals from the color 
separation filter are in the 4:4:4 format. So, three quarters 
of chrominance data become unnecessary when the output signals 
from the color separation filter are input to a device which is 
predicated on image compression of the 4:2:0 or 4:1:1 format. 
Moreover, each luminance signal is formed from four pixels. For 
example, when Y(0,0) and Y(0,1) shown in Figure 2 (b) are formed, 
since G(1,0) and Ye(l,l) are used for both of them, the 
luminance signals are not simply sampling data and are regarded 
as being passed through a low-pass filter in both of the 
vertical and horizontal directions. Therefore, the resolution 
is degraded as compared with a 3CCD type solid-state image 
pickup device performing pixel by pixel sampling. Also each 
color-difference signal is formed from adjacent four pixels and, 
therefore, it is not simple sampling data but is regarded as 
being passed through a low-pass filter in both of the vertical 
and horizontal directions, whereby the resolution is degraded as 
well . 

In order to solve these problems, in a solid-state color 
image pickup apparatus according to the present invention, a 
color separation filter placed on the surface of a solid-state 
image pickup device has an arrangement pattern of four pixels 
corresponding to two whole-color-pass filters, one cyan-pass 



filter, and one yellow-pass filter, and this arrangement pattern 
is repeated. The color separation filter so constructed outputs 
four pieces of luminance data, and two kinds of chrominance data 
each by one. Further, when converting them to luminance signals 
and color-difference signals, the relationship between a pixel 
for which an image of its subject' is to be formed and its 
peripheral pixels is obtained by correlation detection, and 
pixels existing in the direction of higher correlation are used 
for operation. 

Moreover, in the solid-state image pickup device according 
to the present invention, a plurality of chrominance signals are 
output from a plurality of color-pass filters. These 
chrominance signals are independent from color to color, and 
when attention is directed to a specific chrominance signal, its 
sampling rate is lower than the sampling rate of the whole 
signal. Therefore, there is a possibility that aliasing occurs 
in each chrominance signal, and the chrominance signal includes 
a frequency component having aliasing. 

Figure 15 shows aliasing in the case where a specific 
chrominance signal is interpolated. In Figure 15, the abscissa 
shows the frequency and 2K indicates the sampling frequency of 
the whole signal, and the ordinate shows the signal amplitude. 
Further, the solid line shows the chrominance signal, and the 
dashed line shows the aliasing. When a color-difference signal 
is composited by interpolation using the chrominance signal 




including a high-frequency component, an aliasing component is 
unfavorably included in the pass band up to K f2 as shown in 
Figure 15. This results in unsatisfactory interpolation 
precision, and causes a pseudo chrominance signal. 

In order to solve the above problem, the solid-state color 
image pickup apparatus according to the present invention is 
provided with a frequency characteristic adjustment means which 
adjusts the frequency characteristic of each chrominance signal 
::Bi outputted from a solid-state image pickup device using a color 
Ni separation filter composed of two whole-color-pass filters, one 
;=y^ cyan-pass filter, and one yellow-pass filter, and a color- 
ly= difference signal is composited by interpolation using the 

chrominance signals having the adjusted frequency characteristic. 
Furthermore, the solid-state color image pickup apparatus 
P of the present invention is provided with an edge decision 

function for correlation detection, whereby gains by which the 
color-difference signals are to be multiplied are decided, and 
the corresponding color-difference signals are multiplied by 
these gains, thereby reducing pseudo chrominance. 

Accordingly, the present invention can provide a solid- 
state color image pickup apparatus with reduced deterioration of 
luminance resolution and reduced pseudo chrominance. 

DISCLOSURE OF THE INVENTION 

To solve the above-described problems, a solid-state color 




image pickup apparatus defined in Claim 1 of the present 
invention comprises a solid-state image pickup device and a 
signal processing circuit. The solid-state image pickup device 
is provided with a color separation filter having one 
arrangement pattern comprising four pixels adjoining vertically 
and horizontally, said color separation filter of this 
arrangement pattern comprising two whole-color-pass filters 
corresponding to two pixels, a cyan-pass filter corresponding to 
one pixel, and a yellow-pass filter corresponding to one pixel, 
^; and repeating the arrangement pattern in the vertical and 
\^ horizontal directions; and means for taking data corresponding 
I""' to each pixel of the color separation filter. The signal 

processing circuit takes four luminance signals and two kinds of 
color-difference signals, for one of the arrangement patterns, 
r: from each pixel data taken out of the solid-state image pickup 
device and, at this time, the signal processing circuit 
generates two of the four luminance signals by using only the 
data from the whole-color-pass filters, and generates the 
remaining two luminance signals by using the data from the 
whole-color-pass filters and the data from peripheral pixels of 
the four pixels adjoining vertically and horizontally, and 
generates the two kinds of color-difference signals by using the 
data from the cyan or yellow-pass filter and the data from the 
peripheral pixels. Since two of the four luminance signals are 
generated from the data of the whole-color-pass filters, the 
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luminance resolution is improved. 

Further, according to the invention defined in Claim 2, in 
the solid-state color image pickup apparatus defined in Claim 1, 
the color separation filter having one arrangement pattern 
comprising vertically and horizontally adjoining four pixels is 
constituted such that the four pixels are vertical two pixels X 
horizontal two pixels, and six signals comprising four luminance 
signals and two kinds of color-difference signals are generated 
from the data taken out of this arrangement pattern, and the six 
signals so generated are output to a device of 4:2:0 format. 
Thereby, the luminance resolution is improved in the device of 
4:2:0 format. 

Further, according to the invention defined in Claim 3, in 
the solid-state color image pickup apparatus defined in Claim 1, 
the color separation filter having one arrangement pattern 
comprising vertically and horizontally adjoining four pixels is 
constituted such that the four pixels are vertical one pixel X 
horizontal four pixels, and six signals comprising four 
luminance signals and two kinds of color-difference signals are 
generated from the data taken out of the arrangement pattern, 
and the six signals so generated are output to a device of 4:1:1 
format. Thereby, the luminance resolution is improved in the 
device of 4:1:1 format. 

Further, according to the invention defined in Claim 4, in 
the solid-state color image pickup apparatus defined in Claim 1, 
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the color separation filter having one arrangement pattern 
comprising vertically and horizontally adjoining four pixels, 
comprises a whole-color-pass filter and a cyan-pass filter for 
the upper two pixels from the left, and a yellow-pass filter and 
a whole-color-pass filter for the lower two pixels from the left, 
and the apparatus further comprises storage means for capturing 
a chrominance signal outputted from each pixel of the solid- 
state image pickup device, and storing it; correlation 
calculation means for calculating the correlation between a 
target pixel to be interpolated and plural pixels in the 
vicinity of the target pixel, said target pixel being any of a 
cyan signal pixel and a yellow signal pixel stored in the 
storage means; and interpolation means for interpolating the 
target pixel in a direction along which the calculated 
correlation is relatively large, and calculating a whole-color- 
pass signal in the position of the target pixel. Since an input 
image is converted to luminance signals after performing 
interpolation between pixels of high correlation, degradation of 
luminance resolution is suppressed. 

Further, according to the invention defined in Claim 5, in 
the solid-state color image pickup apparatus defined in Claim 4, 
the correlation calculation means calculates the correlation 
between the target pixel and the pixels in the vicinity of the 
target pixel, including the target pixel, in the horizontal or 
vertical direction. Thereby, degradation of luminance 




resolution is suppressed in the vertical direction and 
horizontal direction. 

Further, according to the invention defined in Claim 6, in 
the solid-state color image pickup apparatus defined in Claim 4, 
the correlation calculation means calculates the correlation 
between the target pixel and the pixels in the vicinity of the 
target pixel, including the target pixel, in the horizontal or 
vertical direction and, further, in the diagonal direction. 
Thereby, degradation of luminance resolution is suppressed in 
the vertical direction, horizontal direction, and diagonal 
direction . 

Further, according to the invention defined in Claim 7, in 
the solid-state color image pickup apparatus defined in Claim 4, 
the correlation calculation means calculates the correlation 
between the target pixel and the pixels in the vicinity of the 
target pixel, including the target pixel, in the horizontal or 
vertical direction and, further, in the upper right direction, 
or lower right direction, or upper left direction, or lower left 
direction. Thereby, degradation of luminance resolution is 
suppressed in the vertical direction, horizontal direction, 
upper right L-shaped direction, lower right L-shaped direction, 
upper left L-shaped direction, and lower left L-shaped direction. 

Further, according to the invention defined in Claim 8, in 
the solid-state color image pickup apparatus defined in Claim 4, 
the correlation calculation means calculates the correlation 



between the target pixel and the pixels in the vicinity of the 
target pixel, including the target pixel, in the horizontal or 
vertical direction, and in the diagonal direction, and further, 
in the upper right direction, or lower right direction, or upper 
left direction, or lower left direction. Thereby, degradation 
of luminance resolution is suppressed in the vertical direction, 
horizontal direction, diagonal direction, upper right L-shaped 
direction, lower right L-shaped direction, upper left L-shaped 
direction, and lower left L-shaped direction. 

Further, according to the invention defined in Claim 9, in 
the solid-state color image pickup apparatus defined in Claim 4, 
the correlation calculation means calculates the correlation by 
performing operation on signals of the same color, between the 
target pixel and the pixels in the vicinity of the target pixel. 
Since the correlation is calculated between signals of the same 
color, the precision of correlation calculation is improved. 

Further, according to the invention defined in Claim 10, 
in the solid-state color image pickup apparatus defined in Claim 
4, the correlation calculation means calculates the correlation 
by performing operation on adjacent pixels being signals of 
different colors, between the target pixel and the pixels in the 
vicinity of the target pixel. Since the correlation is 
calculated between the target pixel and a pixel closest to the 
target pixel, even when these pixels are signals of different 
colors, the precision of correlation calculation is improved. 



Further, according to the invention defined in Claim 11, 
in the solid-state color dLmage pickup apparatus defined in Claim 
4, the interpolation means performs interpolation using only- 
signals of the same color as that of a chrominance signal to be 
generated, in the vicinity of the target pixel, without using 
the chrominance signal of the target pixel in the direction 
where the correlation calculated by the correlation calculation 
means is relatively large. Thereby, the interpolation precision 
1 is improved, and the luminance resolution is improved. 

Further, according to the invention defined in Claim 12, 
in the solid-state color image pickup apparatus defined in Claim 
4, the interpolation means calculates a shortage of a 
chrominance signal to be generated, from the pixels in the 
vicinity of the target pixel, using the chrominance signal of 
the target pixel in the direction where the correlation 
calculated by the correlation calculation means is relatively 
large, and performs interpolation on the shortage. Since only 
the shortage of chrominance component is interpolated while 
other components are interpolated using the chrominance signal 
component at the target pixel point, the interpolation precision 
is improved and the luminance resolution is prevented from being 
degraded. 

Further, according to the invention defined in Claim 13, 
in the solid-state color* image pickup apparatus defined in any 
of Claims 5 to 10, when the correlation calculated by the 
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correlation calculation means is smaller than a given threshold^ 
said interpolation means reduces the gain of the color- 
difference signal corresponding to the pixel. Thereby, a pseudo 
chrominance signal generated at the edge of the luminance signal 
is reduced. 

Further, according to the invention defined in Claim 14, 
in the solid-state color image pickup apparatus defined in any 
of Claims 5 to 10, when the correlation calculated by the 
correlation calculation means is smaller than a given threshold, 
said interpolation means reduces the gain of color-difference 
signal corresponding to the pixel, stepwise, according to the 
correlation. Thereby, a pseudo chrominance signal generated at 
the edge of the luminance signal is adaptively reduced. 

Further, according to the invention defined in Claim 15, 
in the solid-state color image pickup apparatus defined in Claim 
4 , the interpolation means is provided with frequency 
characteristic adjustment means for adjusting the frequency 
characteristic of each chrominance signal outputted from the 
solid-state image pickup device, and the interpolation means 
interpolates and composites a color-difference signal by using 
the chrominance signal whose frequency characteristic is 
adjusted. Thereby, a pseudo chrominance signal, which appears 
when the color-difference signal is composited by interpolation 
using the chrominance signal containing high-frequency 
components, is reduced. 




Further/ according to the invention defined in Claim 16, 
in the solid-state color image pickup apparatus defined in Claim 
5, the interpolation means is provided with frequency 
characteristic adjustment means for adjusting the frequency 
characteristic of each chrominance signal outputted from the 
solid-state image pickup device, and the interpolation means 
interpolates and composites an R-Y color-difference signal in 
the position of the cyan-pass filter, and a B-Y color-difference 
:;yi signal in the position of the yellow-pass filter, by using the 
Ml chrominance signal whose frequency characteristic is adjusted. 
Thereby, a pseudo chrominance signal, which appears when the 
color-difference signal is composited by interpolation using the 
P chrominance signal containing high-frequency components, is 
reduced. 

Further, according to the invention defined in Claim 17, 
in the solid-state color image pickup apparatus defined in Claim 
15, the interpolation means decides the correlation direction 
according to the correlation calculated by the correlation 
calculation means, performs frequency characteristic adjustment 
when there is a direction where the correlation is relatively 
large, and does not perform frequency characteristic adjustment 
when there is no direction where the correlation is relatively 
large. Thereby, the frequency component is retained when the 
color-difference signal is composited by interpolation using the 
chrominance signal having no correlation direction, whereby the 




reproducibility of the color of image is maintained. 

Further^ according to the invention defined in Claim 18, 
in the solid-state color image pickup apparatus defined in Claim 
16, the interpolation means decides the correlation direction 
according to the correlation calculated by the correlation 
calculation means, performs frequency characteristic adjustment 
when there is a direction where the correlation is relatively 
large, and does not perform frequency characteristic adjustment 
when there is no direction where the correlation is relatively 
large. Thereby, the frequency component is retained when the 
color-difference signal is composited by interpolation using the 
chrominance signal having no correlation direction, whereby the 
reproducibility of the color of image is maintained. 

As described above, a solid-state color image pickup 
apparatus according to the present invention is provided with a 
solid-state image pickup device having, at its surface, a color 
separation filter comprising two whole-color-pass filters, one 
cyan-pass filter, and one yellow-pass filter corresponding to 
vertically and horizontally adjoining four pixels, and having a 
repetitive pattern of the four pixels; and a circuit for 
extracting four pieces of luminance data and two pieces of 
chrominance data from the four pixels as the repetitive pattern. 
Thereby, a superior solid-state color image pickup device having 
high luminance resolution and no degradation in the chrominance 
resolution is realized. Further, the above-mentioned apparatus 



17 



is further provided with means for capturing signals outputted 
from the respective pixels; correlation calculation means for 
calculating the correlation between a target pixel to be 
interpolated and plural pixels in the vicinity of the target 
pixels said target pixel being any of a cyan signal pixel and a 
yellow signal pixel stored in the storage means; and means for 
performing interpolation in a direction along which the 
calculated correlation is relatively large ^ and calculating a 
whole-color-pass signal in the position of the target pixel. 
Thereby, it is possible to provide a solid-state color image 
pickup device which is able to reduce degradation in the 
luminance resolution, and realize the process of suppressing 
pseudo chrominance signals that appear at edges of the luminance 
signals, without adding a large-scale processing circuit. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1(a) is a block diagram illustrating a solid-state 
color image pickup apparatus according to the present invention. 

Figure 1(b) is a diagram illustrating a pattern of a color 
separation filter placed on the solid-state image pickup device 
shown in figure 1(a). 

Figure 1(c) is a diagram illustrating a pattern of a color 
separation filter placed on the solid-state image pickup device 
shown in figure 1(a). 

Figure 2(a) is a block diagram illustrating the 




conventional solid-state color image pickup apparatus. 

Figure 2(b) is a diagram illustrating a pattern of a color 
separation filter placed on the solid-state image pickup device 
shown in figure 2(a). 

Figure 3(a) is a diagram illustrating the positions of 
luminance and color-difference signals in the 4:2:0 format, from 
a solid-state color image pickup apparatus according to a first 
embodiment of the present invention. 

Figure 3(b) is a diagram illustrating the positions of 
luminance and color-difference signals in the 4:1:1 fo^nnat, from 
a solid-state color image pickup apparatus according to a second 
embodiment of the present invention. 

Figure 4 is a diagram illustrating an arrangement pattern 
of a color separation filter suitable for data output in the 
4:2:0 format, according to the first embodiment of the present 
invention . 

Figure 5 is a diagram illustrating an arrangement pattern 
of a color separation filter suitable for data output in the 
4:1:1 format, according to the second embodiment of the present 
invention . 

Figures 6(a) and 6(b) are diagram for explaining a solid- 
state color image pickup apparatus according to third to seventh 
embodiments of the present invention, wherein figure 6(a) shows 
the structure of the solid-state color image pickup apparatus, 
and figure 6(b) shows the structure of a color-separation filter 
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placed on the solid-state image pickup device • 

Figure 7 is a diagram for explaining calculation of 
correlation, correlation directions (vertical and horizontal 
directions), and interpolation according to a sixth embodiment 
of the present invention. 

Figure 8 is a diagram for explaining calculation of 
correlation, correlation directions (diagonal direction toward 
lower right and diagonal direction toward lower left), and 
interpolation according to a eighth embodiment of the present 
invention . 

Figure 9 is a diagram for explaining calculation of 
correlation, correlation directions (four L-shaped directions), 
and interpolation according to a fifth embodiment of the present 
invention. 

Figure 10 is a diagram for explaining the relationship 
between the correlation and the gain by which the color- 
difference signal is multiplied, according to a ninth embodiment 
of the present invention. 

Figure 11 is a diagram for explaining the relationship 
between the correlation and the gain by which the color- 
difference signal is multiplied, according to the ninth 
embodiment of the present invention. 

Figure 12 is a diagram illustrating the structure of a 
solid-state color image pickup apparatus according to a tenth 
embodiment of the present invention. 
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Figure 13 is a diagram for explaining the operation for 
adjusting the frequency characteristic, according to the tenth 
embodiment of the present invention. 

Figure 14 is a diagram for explaining the operation for 
adjusting the frequency characteristic, according to the tenth 
embodiment of the present invention. 

Figure 15 is a diagram for explaining the operation for 
adjusting the frequency characteristic, according to the tenth 
embodiment of the present invention. 

Figure 16 is a diagram illustrating the structure of a 
solid-state color image pickup apparatus according to an 
eleventh embodiment of the present invention. 

BEST MODE TO EXECUTE THE INVENTION 

Hereinafter, embodiments of the present invention will be 
described. 
Embodiment: 1 . 

Hereinafter, a first embodiment corresponding to Claim 1 
and Claim 2 of the present invention will be described. 

Figure 1(a) illustrates a solid-state color image pickup 
apparatus according to the first embodiment of the invention. 
In the figure, reference numeral 1 denotes an optical system for 
forming an image of a subject on the surface of a solid-state 
image pickup, and it is composed of lenses and the like. 
Reference numeral 2 denotes a solid-state image pickup device 



with a color separation filter, for converting the subject image 
(optical image) so formed to an image signal (electric signal). 
Reference numeral 3 denotes an AD converter for converting the 
image signal obtained by the solid-state image pickup device to 
a digital image signal. Reference numeral 4 denotes an image 
signal processing circuit for converting the digital image 
signal obtained by the AD converter to luminance signals and 
color-difference signals . 

Figure 1(b) illustrates the color separation filter placed 
on the surface of the solid-state image pickup device 2 shown in 
figure 1(a). This filter has a repetitive pattern comprising 
vertical two pixels X horizontal two pixels. In the color 
separation filter, a whole-color-pass filter and a cyan-pass 
filter are arranged from the left for the upper two pixels, and 
a yellow-pass filter and a whole-color-pass filter are arranged 
from the left for the lower two pixels . 

Figure 3(a) shows input /output signals to/from the image 
signal processing circuit 4 shown in Figure 1(a) according to 
the first embodiment. 

In Figure 1(a), an image of the subject is formed on the 
surface of the solid-state image pickup device through the 
optical system 1, and the subject image (optical image) so 
formed is converted to an image signal (electric signal) by the 
solid-state image pickup device 2 with the color separation 
filter. Then, the image signal obtained from the solid-state 



image pickup device is converted to a digital image signal by 
the AD converter 3, and the digital image signal obtained from 
the AD converter is converted to luminance signals and color- 
difference signals by the image signal processing circuit 4. 
Further, the color separation filter attached to the solid-state 
image pickup device 2 is constructed such that it repeats a 
pattern of vertical two pixels X horizontal two pixels as shown 
in figure 1(b), and the color separation filter having this 
pattern is composed of a whole-color-pass filter and a cyan-pass 
filter for the upper two pixels from the left, and a yellow-pass 
filter and a whole-color-pass filter for the lower two pixels 
from the left, that is, it has two whole-color-pass filters for 
;i the two pixels among the four pixels, and one cyan-pass filter 
, and one yellow-pass filter for the remaining two pixels, 

respectively. In this case, four image signals are obtained 
from the solid-state image pickup device: two pieces of whole 
color data, one piece of cyan data, and one piece of yellow data. 
These four pieces of data are subjected to matrix operation to 
obtain four luminance signals, one R-Y color-difference signal, 
and one B-Y color-difference signal outputted from the image 
signal processing circuit 4. 

Hereinafter, a description will be given of the operation 
of converting the digital image signal to luminance signals and 
color-difference signals by the image signal processing circuit 
4, with reference to figure 3(a). 
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The respective color-pass filters are represented by the 
primary colors of light (red (R), green (G), blue (B) ) as 
follows: W=R+G+G, Cy=G+B, and Ye=R+G. With respect to a 
luminance signal comprising all of the R^ G, B components and 
located in a position where the whole-color-pass filter exists, 
this luminance signal is generated from only the signal from the 
whole-color-pass filter which is pure sampling data, and it is 
approximated as follows. 

Y(h,v)=aXW(h,v) 
where a is a factor to adjust the dynamic range, and h+v is 
always an even number in the example of Figure 3(a), 

Further, a luminance signal Y in a position where no 
whole-color-pass filter exists is generated using data of 
peripheral pixels as well, and it approximates as follows in a 
simple method. 

Y(h,v)=aX ( (W(h-l,v)+W(h+l,v)+W(h,v-l)+W(h,v+l) ) ^4) 
where a is a factor to adjust the dynamic range, and h+v is 
always an odd number in the example of Figure 3(a). 

Further, a luminance signal Y in a position where the 
cyan-pass filter exists is generated by interpolation from 
peripheral pixels, utilizing the chrominance data in this 
position which is pure sampling data, since Cy has no R 
component among the luminance signal components, and it is 
approximated as follows . 

R(h,v)=aX (w(h-l ,v)+w(h+l , v)-l-w(h, v-1 )+w(h,v+l ) ^4 
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-bX (Cy(h,v) X4+Cy(h-2,v)+Cy(h+2,v)+Cy(h,v-2)+Cy(h,v+2) ) ^8 
Y(h,v)=bXCy (h, v)-*-R(h,v) 

A luminance signal Y in a position where the yellow-pass 
filter exists is generated by interpolation from peripheral 
pixels because Ye has no B component, and it is approximated as 
follows . 

B(h,v)=aX (w(h-l,v)+w(h+l,v)+w(h,v-l)+w(h,v+l) ) 

-cX (Ye(h,v) X4+Ye(h-2,v)+Ye(h+2,v)+Ye(h,v-2)+Ye(h,v+2) ) ^8 
Y(h,v)=cXYe( h,v)+B(h,v) 

where b and c are factors to adjust the dynamic range, h+v is 
always an odd number in the example of Figure 3 , h and v in the 
position where the cyan-pass filter exists are an odd number and 
an even number, respectively, and h and v in the position where 
the yellow-pass filter exists are an even number and an odd 
number , respectively . 

The luminance signal Y obtained from the Cy and Ye pixels 
is not regarded as being obtained from pure sampling data by the 
solid-state image pickup device, because the R and B components 
are generated by interpolation using the data of the peripheral 
pixels. However, the G+B component for the Cy pixel and the R+G 
component for the Ye pixel remain as pure sampling data, and the 
R and G components to be interpolated are only one third of the 
luminance signal Y at the maximum and, therefore, these 
components do not adversely affect the luminance signal Y, 
resulting in the luminance signal Y maintaining high resolution. 



Furthermore^ with respect to the color-difference signals 
(R-Y, B-Y), one piece of data is taken out for each color- 
difference signal for four luminance signals, and it is 
converted to G, B using, as a pattern, vertical two pixels X 
horizontal two pixels corresponding to the luminance signals, in 
a simple calculation method. The R and B components for 
conversion to the color-difference signals are represented as 
follows . 

R(h,v)=aXW( (h div 2)*2,(v div 2 ) *2 ) -b X Cy ( ( h div 2)*2+l, 
(v div 2)*2) 

B(h,v)=aXW( (h div 2)*2+l,(v div 2 ) *2+l ) -c X Ye ( ( h div 2)*2, 
(V div 2)*2+l ) 
Further, the G component is represented as follows. 

G(h,v)=aX (W( (h div 2)*2,(v div 2)*2)+W((h div 2)*2+l, 
(V div 2)*2+l) ) ^2-R(h,v)-B(h,v) 
According to these R, G, B, the color-difference signals are 
approximated as follows . 

R-Y(h,v)=2 XR(h,v)-G(h,v) 
B-Y(h,v)=2 XB(h,v)-G{h,v) 
where a, b, and c are factors to adjust the dynamic range, div 
means a calculation which takes only the quotient obtained by 
division of integers and omits the remainder, and * means 
multiplication . 

A pair of color-difference signals so obtained have 
improved chrominance resolution because the output of the solid- 




state image pickup device is not used for adjacent other color- 
difference signals. 

Through the above-mentioned operation^ as shown in figure 
3(a), two kinds of color-difference signals R-Y and B-Y are 
obtained, each by one, for four luminance signals Y 
corresponding to vertical two pixels X horizontal two pixels , 
and these signals are suitable as input signals to a device of 
the 4:2:0 format . 

In addition, when the transmissivity of the whole-color- 
pass filter is set at 0.3 for R, 0.59 for B, and 0.11 for G, a 
pure luminance signal is obtained because the ratio of the 
transmissivities is equal to the mixing ratio of primary colors 
of the luminance signal Y, resulting in further improved 
resolution. That is, usually, the respective color-pass filters 
are expressed as follows with the primary color components (red, 
green, blue (R, G, B)): W=R+G+B, Cy=G+B, Ye=R+G. However, when 
the above-described setting is carried out, the RGB mixing ratio 
of the luminance signal Y is represented by 

Y=:0 . 30R+0 . 59G+0 . IIB 
and the transmissivities of the respective color-pass filters 
used for this first embodiment are adjusted as follows. 

W=0 .30R+0.59G+0.11B 
Cy=(0.5 9G+0.11B) 0 . 7 
Ye=(0.3 0R+0.5 9G) ^0.89 
At this time, a luminance signal Y in a position where the 
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whole-color-pass filter exists is generated from the signal 
outputted from the whole-color-pass filter, which becomes pure 
sampling data, and this luminance signal is represented by 

Y(h,v)=aXW(h,v) 

where a is a factor for adjusting the dynamic range, and h+v is 
always an even number in the example of Figure 3 (a). Further, 
a luminance signal Y in a position where no whole-color-pass 
filter exists is generated using data of peripheral pixels, in a 
simple method as follows. 

Y(h,v)=aX ( (W(h-1 ,v)+W(h+l,v)+W(h,v-l )+W(h, v+1 ) ) -^4) 
Further, utilizing the chrominance data in this position which 
is pure sampling data, a luminance signal Y in a position where 
the cyan-pass filter exists is generated as follows. 

Y(h,v)=aX ( (W(h-l,v)+W(h+l,v)+W(h,v-l)+W(h,v+l) )-r4) 
+0 . 7 XbX (Cy(h-2,v) -=-2- ( Cy ( h+2 , v )+Cy ( h, v-2 ) 
+Cy(h,v+2)+Cy(h,v) ) ^8) 

Further, a luminance signal Y in a position where the 
yellow-pass filter exists may be generated as follows. 

Y(h,v)=aX ( (W(h-1 ,v)+W(h+l,v)+W(h,v-l )+W(h, v+1 ) ) ^4) 
+0.7 XcX (Ye(h-2,v) ~ 2- ( Ye( h+2 , v) +Ye ( h, v-2 ) 
+Ye ( h , vH-2 ) +Ye ( h , v ) ) 8 ) 
where b and c are factors for adjusting the dynamic range, and 
h+v is always an odd number in the example of Figure 3 (a), h 
and V in the position where the cyan-pass filter exists are an 
odd number and an even number, respectively, and h and v in the 



• as • 



position where the yellow-pass filter exists are an even number 
and an odd number, respectively. 

Moreover, with respect to the color-difference signals (R- 
Y, B-Y), one piece of data is taken out for each color- 
difference signal for four luminance signals, and the color- 
difference signals are obtained using, as a pattern, vertical 
two pixels X horizontal two pixels corresponding to the 
luminance signals, in a relatively simple calculation method as 
follows . 

R-Y(h,v)=0.7-f0.3 X (aX (W( (h div 2)*2,(v div 2)*2) 

+W((h div 2)*2 + l,(v div 2 )*2+l ) ) -=-2-bXCy( (h div 2)*2+l, 
(V div 2 )*2) ) 

B-Y(h,v)=0.89-f O.llX (aX (W( (h div 2)*2,(v div 2)*2) 

+W((h div 2)*2+l,(v div 2 )*2+l ) ) ^2-bX Ye( (h div 2)*2, 
(V div 2 )*2+l ) ) 

Alternatively, considering the sampling positions of the color- 
difference signals, the color difference signals may be obtained 
as follows, 

R-Y(h,v)=0.7^0.3X (aX (W( (h div 2)*2,(v div 2)*2) 

+W((h div 2)*2+l,(v div 2 ) *2+l ) ) 2-b X ( Cy ( ( h div 2)*2+l, 
(V div 2)*2) X2+Cy( (h div 2)*2-l,(v div 2)*2) 
+Cy((h div 2)*2+l,(v div 2)*2+2))-^4) 

B-Y(h,v)=0.89^0.11X (aX (W( (h div 2)*2,(v div 2)*2) 

+W((h div 2)*2 + l,(v div 2 ) *2 + l ) ) ^ 2-b X ( Ye( ( h div 2)*2, 
(V div 2)*2 + l)X2 Ye((h div 2)*2,(v div 2)*2-l) 
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+Ye((h div 2)*2+2,(v div 2)*2+l))-f4 
In this first embodiment/ the whole-color-pass filters are 
arranged checkerwise. However, as shown in figure 4(a)/ in a 
repetitive pattern comprising vertical two pixels X horizontal 
two pixels, two whole-color-pass filters may be adjacent to each 
other in the horizontal direction, by arranging the two whole- 
color-pass filters for the upper two pixels, and a cyan-pass 
filter and a yellow-pass filter for the lower two pixels. In 
;:B this case, the horizontal resolution is improved. Further, as 
H shown in figure 4(b), two whole-color-pass filters may be 

adjacent to each other in the vertical direction, by arranging a 
w whole-color-pass filter and a cyan-pass filter for the upper two 

pixels, and a whole-color-pass filter and a yellow-pass filter 
tU for the lower two pixels. In this case, the vertical resolution 
v3 is improved. Further, the arrangement of the filters 

corresponding to the four pixels may be varied from pattern to 
pattern with the same effect as mentioned above. 

Further, while in this first embodiment the color 
separation filter outputs signals in the 4:2:0 format, it can 
output signals in the 4:4:4, 4:2:2, or 4:4:1 format because the 
R, G, B components can be placed in any position by 
appropriately utilizing the peripheral pixels, although the 
chrominance resolution is somewhat degraded when performing 
conversion to the color-difference signals. 

Further, while in this first embodiment the cyan-pass 



• 30 • 



filter and the yellow-pass filter are employed as filters other 
than the whole-color-pass filter, these filters may be a red- 
pass filter and a blue-pass filter. In this case, the 
calculation is simplified because there is no necessity of 
taking the R and B components from the filters. However, since 
the red-pass filter and the blue-pass filter include no green 
components, green components should be supplied from the 
peripheral pixels, whereby the luminance resolution is degraded. 

Furthermore, in this first embodiment, the color 
separation filter having a pattern of vertical two pixels X 
horizontal two pixels is composed of a whole-color-pass filter 
and a cyan-pass filter for the upper two pixels from the left, 
and a yellow-pass filter and a whole-color-pass filter for the 
lower two pixels from the left. However, this arrangement may 
be changed without changing the colors (i.e., cyan and yellow) 
of the color-pass filters, or the combination of the colors of 
the color-pass filters may be changed from cyan and yellow to 
cyan and magenta, magenta and yellow, red and blue, red and 
green, or green and blue. 
Embodimen-b 2 . 

Hereinafter, a second embodiment corresponding to Claim 1 
and Claim 3 of the present invention will be described. 

Figure 1(c) shows a color separation filter attached to 
the surface of the solid-state image pickup device 2 shown in 
figure 1(a). This filter has a repetitive pattern of vertical 



one pixel X horizontal four pixels^ and comprises a whole- 
color-pass filter, a cyan-pass filter, a whole-color-pass filter, 
and a yellow-pass filter which are arranged in this order from 
the left of the pattern. 

Figure 3(b) shows input/output signals to/from the image 
signal processing circuit 4 shown in figure 1(a). 

In figure 1(a), an image of a subject is formed at the 
surface of the solid-state image pickup device through the 
optical system 1, and the subject image (optical image) so 
formed is converted to an image signal (electric signal) by the 
solid-state image pickup device 2 with the color separation 
filter. Then, the image signal obtained from the solid-state 
image pickup device is converted to a digital image signal by 
the AD converter 3, and the digital image signal obtained from 
the AD converter is converted to luminance signals and color- 
difference signals by the image signal processing circuit 4. 
Further, as shown in figure 1(c), the color separation filter 
attached to the solid-state image pickup device 2 has a 
repetitive pattern of vertical one pixel X horizontal four 
pixels, and comprises, from the left, a whole-color-pass filter, 
a cyan-pass filter, a whole-color-pass filter, and a yellow-pass 
filter. That is, the color separation filter has two whole- 
color-pass filters for two pixels among four pixels, and a cyan- 
pass filter and a yellow-pass filter for the remaining two 
pixels. In this case, an image signal obtained from the solid- 
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state image pickup device comprises four pieces of data, i.e., 
two pieces of whole color data, one piece of cyan data, and one 
piece of yellow data. The image signal processing circuit 4 
subjects these four pieces of data to matrix calculation, and 
outputs four luminance signals, one R-Y color-difference signal, 
and one B-Y color difference signal. 

Hereinafter, a description will be given of the operation 
for converting the digital image signal to luminance signals and 
color-difference signals by the image signal processing circuit, 
£ with reference to figure 3(b). 

jyi The respective color-pass filters are represented by the 

I" primary color components of light (red (R), green (G), blue (B)) 
m as follows: W=R+G+B^ Cy=G-i-B, and Ye=R+G. A luminance signal Y 
ffl in a position where the whole-color-pass filter exists, whose 
U: luminance data contains all of the R, G, B components, is 
generated from a signal outputted from the whole-color-pass 
filter which becomes pure sampling data, and it is approximated 
as follows. 

Y(h,v)=aXW(h,v) 

where a is a factor to adjust the dynamic range, and h is always 
an even number in the example of Figure 3 (b). 

Further, a luminance signal in a position where no whole- 
color-pass filter exists is generated using data of peripheral 
pixels, and it is approximated as follows in a simple generation 
method. 
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Y(h,v)=aX ( (W(h-1, v)+W(h+l,V) ) ^2) 
where a is a factor to adjust the dynamic range, and h is always 
an odd number in the example of Figure 3 (b). 

Moreover, effectively utilizing the chrominance data as 
pure sampling data in the corresponding position, a luminance 
signal Y in a position where the cyan-pass filter exists is 
generated by interpolation from peripheral pixels because the Cy 
has no R components, and it is approximated as follows. 

;iQ R(h,v)=aX ( (w(h-l ,v)+w(h+l,v) ) X 2+w( h-1 , v-1 ) +w( h+1 , v-1 ) 

■4 +w(h-l,v+l)+w(h+l,v+l) )-h8-bX (Cy(h,v) X2+Cy(h,v-1) 

;D +Cy(h,v+1) ) -=-4 

W Y(h,v)=bXCy(h,v)+R(h,v) 

Q Further, a luminance signal Y in a position where the 

yellow-pass filter exists is generated by interpolation from 

P peripheral pixels because the Ye has no B components, and it is 
approximated as follows . 

B(h,v)=aX ( (w(h-l ,v)+w(h+l,v) ) X 2+w( h-1 , v-1 ) +w( h+1 , v-1 ) 

+w(h-l ,v+l )+w( h+1 , v+1 ) )^8-cX (Ye(h,v) X2+Ye(h,v-1) 
+Ye(h, v+1 ) -r4 

Y(h,v)=cXYe(h,v)-HB( h,v) 

where b and c are factors to adjust the dynamic range, and mod 
means a calculation to take only the remainder obtained by 
division of integers, whereby h in the position where the cyan- 
pass filter exists becomes (h mod 4)=1, and h in the position 
where the yellow-pass filter exists becomes (h mod 4) =3. 
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The R and B components required for obtaining the above- 
described luminance signal Y from the Cy and Ye pixels are 
generated by interpolation using the data of the peripheral 
pixels and, therefore, the luminance signal Y is not regarded as 
being obtained from pure sampling data by the solid-state image 
pickup device. However, the G+B components for the Cy pixels 
and the R+G components for the Ye pixels remain as pure sampling 
data, and the R and G components to be interpolated are one 
third of the luminance signal Y at the maximum and do not 
adversary affect the luminance signal Y, resulting in the 
luminance signal Y maintaining high resolution. 

With respect to the color-difference signals (R-Y, B-Y) , 
one piece of data is taken out for each color-difference signal 
for four luminance signals and, in a simple calculation method, 
it is converted to R, G, B using, as a pattern, vertical one 
pixel X horizontal four pixels corresponding to the luminance 
signals. The R and B components for this conversion are 
represented by 

R(h,v)=aXW( (h div 4 )*4,v)-bXCy( (h div 4)*4 + l,v) 

B(h,v)=aXW( (h div 4 )*4+2 ,v) -c X Ye( (h div 4)*4+3,v) 
and the G component is represented by 
G(h,v)=aXW( (h div 4)*4,v)+W((h div 4 ) *4+2 , v) -i- 2-R( h, v ) -B ( h, v ) 

Using these R, G, B components, the color-difference 
signals are approximated as follows. 

R-Y(h,v)=2 XR(h,v)-G(h,v) 




B-Y(h,v)=2 XB(h,v)-G(h^v) 
where a, b and c are factors to adjust the dynamic range. 

A pair of color-difference signals so obtained have 
improved color resolution because the output of the solid-state 
image pickup device is not used for adjacent other color- 
difference signals. 

Through the above-mentioned operation, as shown in figure 
3(b), two kinds of color-difference signals R-Y and B-Y are 
obtained, each by one, for the four luminance signals Y 
corresponding to vertical one pixel X horizontal four pixels, 
and these signals are suitable as input signals to a device 
employing the 4:1:1 format . 

In addition, when the transmissivity of the whole-color- 
pass filter is set at 0.3 for R, 0.59 for B, and 0.11 for G, a 
pure luminance signal is obtained because the ratio of the 
transmissivities is equal to the mixing ratio of primary colors 
of the luminance signal Y, resulting in further improved 
resolution. The transmissivities can be obtained by a 
calculation method similar to that described for the first 
embodiment although the matrix is different from that of the 
first embodiment. 

Further, as described for the first embodiment, not only 
4:1:1 output but also 4:4:4, 4:2:2, or 4:2:0 output can be 
achieved because the R, G, B components can be placed in any 
position by appropriately utilizing the peripheral pixels 




although the chrominance resolution is somewhat degraded when 
performing conversion to the difference-color signals. 

As shown in figure 5(a), the color-pass filters may be 
shifted by one pixel when the pattern of vertical one pixel X 
horizontal four pixels shown in figure 1(c) or 3(b) is repeated 
in the vertical direction so as to arrange the whole-color-pass 
filters checkerwise. In this case, the resolution of the 
luminance signal in the diagonal direction can be improved. 
Further, as shown in figure 5(b), the whole-color-pass filters 
and the color-pass filters other than the whole-color pass 
filters shown in figure 1(c) may be interchanged and, further, 
the cyan-pass filter and the yellow-pass filter may be 
interchanged, providing a repetitive pattern of 8 pixels. This 
filter arrangement provides improved chrominance resolution and 
is suitable for both of the 4:1:1 format (vertical one pixel X 
horizontal four pixels) and the 4:2:0 format (vertical two 
pixels X horizontal two pixels). 

In this second embodiment, the color separation filter 
having a pattern of vertical one pixel X horizontal four pixels 
is composed of, from the left, a whole-color-pass filter, a 
cyan-pass filter, a whole-color-pass filter, and a yellow-pass 
filter. However, the arrangement of these filters may be 
changed without changing the colors (i.e., cyan and yellows) of 
the color-pass filters, or the combination of the colors of the 
color-pass filters may be changed from cyan and yellow to cyan 



and magenta, magenta and yellow, red and blue, red and green^ or 
green and blue. In the repetitive pattern of the color-pass 
filters corresponding to four pixels, when providing two whole- 
color-pass filters and two kinds of color-pass filters 
transparent to colors other than the whole color, the same 
effects as mentioned above are achieved. Further, four patterns 
of the color separation filter, each comprising vertical one 
pixel X horizontal four pixels, are provided in the vertical 
direction, and the arrangements of the color-pass filters in 
these four patterns may be different from each other. 
Embodiment 3 . 

Hereinafter, a third embodiment corresponding to Claims 4, 
5, 9, and 12 of the present invention will be described with 
reference to figures 6 and 7 . 

In figure 6(a), reference numeral 1 denotes an optical 
system for making an image of a subject on the surface of a 
solid-state image pickup device, and it is composed of lenses 
and the like. Reference numeral 2 denotes a solid-state image 
pickup device with a color separation filter, for converting the 
subject image (optical image) so formed to an image signal 
(electric signal). Reference numeral 3 denotes an AD converter 
for converting the image signal obtained from the solid-state 
image pickup device 2 to a digital image signal. Reference 
numeral 5 denotes a storage circuit for storing one frame of the 
digital image signal obtained in the AD converter 3. Reference 
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numeral 6 denotes a correlation calculation circuit for 
calculating the correlation between an arbitrary pixel of the 
digital image signal stored in the storage circuit 5, and 
peripheral pixels. Reference numeral 7 denotes an interpolation 
circuit for performing interpolation according to the 
correlation calculated by the correlation calculation circuit 6 
to output luminance signals and color-difference signals. The 
luminance signals and the color-difference signals are generated 
by the optical system 1, the solid-state image pickup device 2 
with the color separation filter, the AD converter 3, the 
storage circuit 5, the correlation calculation circuit 6, and 
the interpolation circuit 7. 

Figure 6(b) shows the structure of the color separation 
filter placed on the solid-state image pickup device 2. This 
color separation filter has, as one unit of arrangement, four 
pixels adjacent to each other in the vertical and horizontal 
directions, and this arrangement unit is composed of a whole- 
color-pass filter and a cyan-pass filter for the upper two 
pixels from the left, and a yellow-pass filter and a whole- 
color-pass filter for the lower two pixels from the left. This 
arrangement unit is repeated in the vertical and horizontal 
directions . 

The W pixel, Cy pixel, and Ye pixel captured in a storage 
circuit (not shown) are expressed by R, G, B components as 
follows: W=(R+G+B)/3, Cy=(G+B)/3, and Ye=(R+B)/3. When W = Y, 




the output signal corresponding to the W pixel can be expressed 
as a luminance signal. With respect to the Cy pixel and Ye 
pixel, the R and B components are obtained by interpolation and 
added to the Cy pixel and Ye pixel, respectively, whereby a 
luminance signal is expressed. Although signals corresponding 
to peripheral pixels are used for interpolation, the peripheral 
pixels used for interpolation are decided by calculating the 
correlation with a target pixel to be interpolated, by the 
correlation calculation circuit 6. Initially, the correlation 
calculating method will be described. 

Figure 7 shows the arrangement of peripheral pixels when a 
cyan pixel Cyn is a target pixel. In figure 7, • and O 
indicate Ye pixels and W pixels which are not used in the 
interpolation process for the pixel Cyn. Assuming that the 
correlation in the vertical direction (®-®' direction in figure 
7) is Vc and the correlation in the horizontal direction (©-(g)' 
direction in figure 7) is He, these correlations Vc and He are 
calculated using the following expressions. 

Vc= I Wu-Wd I + I Cyu-Cyn | + | Cyd-Cyn | ( 1 ) 

Hc= I Wl-Wr I + I Cyl-Cyn | + | Cyr-Cyn | ( 2 ) 

Using the result of the calculation, the correlation 
direction is decided according to the following conditional 
expressions . 

Vc+Th < He (3) 
Hc+Th < Vc (4) 




wherein Th is a threshold^ and this is a specific constant. The 
correlation direction is decided as the vertical direction when 
the expression (3) holds and as the horizontal direction when 
the expression (4) holds. When none of the expressions (3) and 
(4) holds, it is decided that there is no correction direction. 

Next, the interpolation process will be described. 

When the correlation direction is the vertical direction, 
a shortage component RCy is calculated using, as pixels for 
interpolation, only the peripheral pixels in the vertical 
direction with respect to the target pixel Cyn, according to the 
following expres s ion . 

RCy=(Wu+Wd) /2-(2*Cyn+Cyu+Cyd) /4 (5) 
When the correlation direction is the horizontal direction, 
a shortage component RCy is calculated using only the peripheral 
pixels in the horizontal direction with respect to the target 
pixel Cyn, according to the following expression. 

RCy= ( Wl+Wr ) / 2 - ( 2*Cy n+Cy 1+Cy r ) / 4 ( 6 ) 

When there is no correlation direction, a shortage 
component RCy is calculated using the peripheral pixels in both 
of the horizontal and vertical directions with respect to the 
target pixel Cyn, according to the following expression. 

RCy=(Wu+Wd+Wl+Wr ) /4- (4*Cyn+Cyu+Cyd+Cyl+Cyr) /8 ( 7 ) 

Using the shortage component RCy obtained by any of the 

expressions (5) to (7), the W component of the target pixel Cyn 

is obtained as follows. 



W • =Cyn+RCy 

For all of target pixels Cyn, W is calculated by the 
above-described operation • 

When the target pixel is a Ye pixel, Cy in the expressions 
(1) and (2) is replaced with Ye to calculate the correlation, 
and Cy in the right sides of the expressions (5) to (7) is 
replaced with Ye to obtain a shortage component BYe. Since the 
shortage component so obtained is not the R component but the B 
component, the W component of the Ye pixel can be obtained by 
W'=Yen+Bye. For all of the target pixels Yen, W is calculated 
by the above-described operation. 

In this interpolation process, the luminance W of the Cy 
pixel and the luminance W* of the Ye pixel are obtained, whereby 
all of the luminance signals are obtained. In this method, the 
signals of the W pixels are used as they are, and the Cy pixels 
and Ye pixels are interpolated using the peripheral pixels 
having the higher correlation with them, whereby degradation in 
resolution is suppressed. 

As described above, according to the third embodiment, the 
correlation between the target pixel to be interpolated and the 
peripheral pixels in the vertical and horizontal directions 
including the target pixel is detected, and the target pixel is 
interpolated using the correlation. Therefore, highly precise 
luminance signals are obtained, and the resolution is prevented 
from being degraded. 



Embodimen'b 4 . 

Hereinafter, a fourth embodiment corresponding to Claim 6 
of the present invention will be described with reference to 
figure 8. 

The construction of this fourth embodiment is identical to 
that of the third embodiment. In this fourth embodiment, the 
correlation calculation circuit 6 calculates the correlation in 
the diagonal direction as well, and the interpolation circuit 7 
performs interpolation using the correlation in the diagonal 
direction . 

Hereinafter, the correlation calculating method will be 
described . 

Figure 8 shows the arrangement of peripheral pixels when a 
cyan pixel Cyn is a target pixel to be interpolated. In figure 
8, # and O indicate Ye pixels and W pixels which are not used 
in the interpolation process for the pixel Cyn. 

In the third embodiment of the invention, only the 
correlations in the ®-®* direction and the @-(D* direction are 
obtained as shown in figure 7. In this fourth embodiment, 
assuming that the correlation in the diagonal direction toward 
the lower right (the direction in figure 8) is Nr, and the 

correlation in the diagonal direction toward the lower left (the 
(D-®* direction in figure 8) is Nl, and these correlations Nr 
and Nl are calculated using the following expressions. 

Nr=| (Wu+Wl)/2-(Wd+Wr)/2|+| (Cyul-Cyn) |+| (Cydr-Cyn) | (8) 



Nl=| (Wu+Wr)/2-(Wd+Wl)/2 |+| (Cydl-Cyn) |+| (Cyur-Cyn) | (9) 
Using the result of this calculation and the Vc and He 
obtained from the expressions (1) and (2)^ the correlation 
direction is decided according to the following conditional 
expressions . 

Vc+Th < min(Hc,Nr^Nl) (10) 

Hc+Th < min(Vc,Nr,Nl) (11) 

Nr+Th < min(Hc,Vc,Nl) (12) 

Nl+Th < min(Hc,Vc,Nr ) (13) 
wherein Th is a threshold (a specific constant)^ and min is a 
function which takes the minimum value among the elements in the 
parentheses. The correlation direction is the vertical 
direction when the expression (10) holds, the horizontal 
direction when the expression (11) holds, the diagonal direction 
toward the lower right when the expression (12) holds, and the 
diagonal direction toward the lower left when the expression 
(13) holds. When none of the expressions (10) to (13) holds, 
there is no correlation direction. 

Next, the interpolation process will be described. 
While in the third embodiment the interpolation process is 
described for the case where the correlation direction is 
decided as the vertical or horizontal direction and the case 
where there is no correlation direction, in this fourth 
embodiment the interpolation process in the case where the 
correlation direction is decided as the diagonal direction will 




be added. 

When the correlation direction is the diagonal direction 
toward the lower rights a shortage component RCy is calculated 
using only the peripheral pixels in the diagonal direction 
toward the lower right with respect to the target pixel Cyn, 
according to the following expression. 

RCy=(Wu+Wd+WH-Wr ) /4- ( 2*Cyn+Cyul+Cydr ) /4 ( 14 ) 

When the correlation direction is the diagonal direction 
;;Qi toward the lower left^ a shortage component RCy is calculated 
Ui using only the peripheral pixels in the diagonal direction 
=;Bi toward the lower left with respect to the target pixel Cyn, 
ly according to the following expression. 

v3 RCy = ( Wu+Wd+Wl+Wr ) / 4 - ( 2 *Cy n+Cy ur +Cy dl ) / 4 (15) 

ry Thereafter, interpolation is performed in like manner as 

11 described for the third embodiment, thereby all of the luminance 
signals are obtained. 

As described above, in this fourth embodiment, since 
interpolation is performed using not only the correlations in 
the vertical and horizontal directions but also the correlations 
in the diagonal directions, degradation of resolution is 
suppressed in the diagonal directions as well as in the vertical 
and horizontal directions. 
Embodiment 5 • 

Next, a fifth embodiment corresponding to Claim 7 of the 
present invention will be described with reference to figure 9. 



45 



The construction of this fifth embodiment is identical to 
that of the third embodiment. In this fifth embodiment, the 
correlation calculation circuit 6 calculates the correlations in 
L-shaped directions as well, and the interpolation circuit 7 
performs interpolation using the correlations in the L-shaped 
directions . 

Initially, the correlation calculation method will be 
described. 

Figure 9 shows the arrangement of peripheral pixels when a 
cyan pixel Cyn is a target pixel. In figure 9, # and O 
indicate Ye pixels and W pixels which are not used in the 
interpolation process for the pixel Cyn. 

In the third embodiment of the invention, only the 
correlations in the (1)-®' direction and the d)-©' direction are 
obtained as shown in figure 7. In this fifth embodiment, 
assuming that the correlation in the upper left L-shaped 
direction (the direction in figure 9) is Lul, the 

correlation in the upper right L-shaped direction (the 
direction) is Lur, the correlation in the lower left L-shaped 
direction (the (2)-®' direction) is Ldl, and the correlation in 
the lower right L-shaped direction (the (§)-(B) ' direction) is Ldr, 
and these correlations are calculated using the following 
expressions . 

Lul= I Wu-Wl I + I Cyu-Cyn | + | Cyl-Cyn | (16) 
Lur= I Wu-Wr I + I Cyu-Cyn | + | Cyr-Cyn | (17) 



Ldl= I Wd-Wl I + I Cyd-Cyn | + 1 Cyl-Cyn | ( 18 ) 

Ldr= I Wd-Wr | + | Cyd-Cyn | + | Cyr-Cyn | (19) 
Using the result of calculation and the Vc and He obtained 
from the expressions (1) and (2), the correlation direction is 
decided according to the following conditional expressions. 
Vc+Th < min(Hc,Lul,Ldl,Lur,Ldr ) (20) 
Hc+Th < min(Vc,Lul,Ldl,Lur,Ldr ) (21) 
Lul+Th < min(Hc,Vc,Ldl,Lur,Ldr ) (22) 
Lur+Th < inin(Hc, Vc,Lul,Ldl,Ldr ) (23) 
Ldl+Th < min(Hc^Vc,Lul,Lur,Ldr ) (24) 
Ldr+Th < inin(Hc, Vc,Lul,Ldl,Lur ) (25) 
where Th is a threshold (a specific constant), and min is a 
function which takes the minimum value among the elements in the 
parentheses. The correlation direction is the vertical 
direction when the expression (20) holds, the horizontal 
direction when the expression (21) holds, the upper left L- 
shaped direction when the expression (22) holds, the upper right 
L-shaped direction when the expression (23) holds, the lower 
left L-shaped direction when the expression (24) holds, and the 
lower right L-shaped direction when the expression (25) holds. 
When none of the expressions (20) to (25) holds, there is no 
correlation direction. 

Next, the interpolation process will be described. 
While in the third embodiment the interpolation process is 
described for the case where the correlation direction is the 



vertical or horizontal direction and the case where there is no 
correlation direction, in this fifth embodiment the 
interpolation process in the case where the correlation 
direction is any of the above-mentioned L-shaped directions will 
be added. 

When the correlation direction is the upper left L-shaped 
direction, a shortage component RCy is calculated using only the 
peripheral pixels in the upper left L-shaped direction with 
respect to the target pixel Cyn, according to the following 
expression. 

RCy = ( Wu+Wl ) / 2 - ( 2 *Cy n+Cyu+Cy 1 ) / 4 (26) 
When the correlation direction is the upper right L-shaped 
direction, a shortage component RCy is calculated using only the 
peripheral pixels in the upper right L-shaped direction with 
respect to the target pixel Cyn, according to the following 
expression. 

RCy=(Wu+Wr)/2-(2*Cyn+Cyu+Cyr)/4 (27) 
When the correlation direction is the lower left L-shaped 
direction, a shortage component RCy is calculated using only the 
peripheral pixels in the lower left L-shaped direction with 
respect to the target pixel Cyn, according to the following 
expression. 

RCy = ( Wd+Wl ) / 2 - ( 2 *Cy n+Cy d+Cy 1 ) / 4 (28) 
When the correlation direction is the lower right L-shaped 
direction, a shortage component RCy is calculated using only the 
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peripheral pixels in the lower right L-shaped direction with 
respect to the target pixel Cyn, according to the following 
expression. 

RCy = ( Wd+Wr ) / 2 - ( 2 *Cy n+Cy d+Cy r ) / 4 (29) 
Thereafter^ interpolation is carried out in like manner as 
described for the third embodiment, whereby all of the luminance 
signals are obtained. 

According to the fifth embodiment, as described above, not 
only the correlations in the vertical and horizontal directions 
but also the correlations in the respective L-shaped directions 
are detected to be used for interpolation. Therefore, 
degradation in resolution is suppressed in the L-shaped 
directions as well as in the vertical and horizontal directions. 
Embodiment 6 . 

Next, a sixth embodiment corresponding to Claim 8 of the 
present invention will be described. 

The construction of this sixth embodiment is fundamentally 
identical to that of the third embodiment. In this sixth 
embodiment, however, the correlation calculation circuit 6 
calculates the correlations in the diagonal directions and the 
L-shaped directions as well as correlations in the vertical and 
horizontal directions, and the interpolation circuit 7 performs 
interpolation using the correlations in the diagonal directions 
and the L-shaped directions as well. 

Initially, the correlation calculation method will be 
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described • 

In the third embodiment, only the correlations Vc and He 
in the ®-®* direction and the @-(D' direction are obtained as 
shown in figure 7. In this sixth embodiment, more correlations 
as follows are obtained in like manner as described for the 
fourth and fifth embodiments: the correlation Nr in the diagonal 
direction toward the lower right (the (1)-®' direction in figure 
8), the correlation Nl in the diagonal direction toward the 
lower left (the (D-@' direction), the correlation Lul in the 
upper left L-shaped direction (the (S)-®' direction in figure 9), 
the correlation Lur in the upper right L-shaped direction (the 
©-(D' direction), the correlation Ldl in the lower left L-shaped 
direction (the (2)-®' direction), and the correlation Ldr in the 
lower right L-shaped direction (the (0)-®' direction). 

Using the results, the correlation direction is decided by 
the following conditional expressions. 



vc+Th 


< min(Hc,Nr, 


Nl,Lul 


, Lur , Ldl 


/Ldr) 


(30) 
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< min(Vc,Nr, 
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(32) 


Nl+Th 


< min(Hc,Vc, 


Nr ,Lul 


, Lur , Ldl 


/Ldr) 


(33) 


Lul+Th 


< min(Hc,Vc 


,Nr,Nl 


, Lur , Ldl 


/Ldr) 


(34) 


Lur+Th 


< inin(Hc,Vc 


,Nr,Nl 


, Lul, Ldl 


/Ldr) 


(35) 


Ldl+Th 


< itiin(Hc,Vc 


,Nr,Nl 


, Lul, Lur 


/Ldr) 


(36) 


Ldr+Th 


< niin(Hc,Vc 


,Nr,Nl 


, Lul , Lur 


/Ldl) 


(37) 



where Th is a threshold (a specific constant), and min is a 
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function which takes the minimum value among the elements in the 
parentheses. The correlation direction is the vertical 
direction when the expression (30) holds, the horizontal 
direction when the expression (31) holds, the diagonal direction 
toward lower right when the expression (32) holds, the diagonal 
direction toward lower left when the expression (33) holds, the 
upper left L-shaped direction when the expression (34) holds, 
the upper right L-shaped direction when the expression (35) 
holds, the lower left L-shaped direction when the expression 

(36) holds, and the lower right L-shaped direction when the 
expression (37) holds. When none of these expressions (30) to 

(37) holds, there is no correlation direction. 

Next, the interpolation process will be described. 

While in the third embodiment the interpolation process is 
described for the case where the correlation direction is the 
vertical or horizontal direction and the case where there is no 
correlation direction, in this sixth embodiment the 
interpolation process in the case where the correlation 
direction is any of the above-mentioned diagonal directions and 
L-shaped directions will be added. 

When the correlation direction is the diagonal direction 
toward the lower right, a shortage component RCy is calculated 
using only the peripheral pixels in the diagonal direction with 
respect to the target pixel Cyn, according to the expression 
(14). When the correlation direction is the diagonal direction 




toward the lower left, a shortage component RCy is calculated 
using only the peripheral pixels in the diagonal direction with 
respect to the target pixel Cyn, according to the expression 
(15). When the correlation direction is the upper left L-shaped 
direction, a shortage component RCy is calculated using only the 
peripheral pixels in the upper left L-shaped direction with 
respect to the target pixel Cyn, according to the expression 
(26). When the correlation direction is the upper right L- 
shaped direction, a shortage component RCy is calculated using 
only the peripheral pixels in the upper right L-shaped direction 
with respect to the target pixel Cyn, according to the 
expression (27). When the correlation direction is the lower 
left L-shaped direction, a shortage component RCy is calculated 
using only the peripheral pixels in the lower left L-shaped 
direction with respect to the target pixel Cyn, according to the 
expression (28). When the correlation direction is the lower 
right L-shaped direction, a shortage component RCy is calculated 
using only the peripheral pixels in the lower right L-shaped 
direction with respect to the target pixel Cyn, according to the 
expression (29). 

Thereafter, interpolation is carried out in like manner as 
described for the third embodiment, whereby all of the luminance 
signals are obtained. 

According to the sixth embodiment, as described above, not 
only the correlations in the vertical and horizontal directions 
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but also the correlations in the respective diagonal directions 
and L-shaped directions are detected to be used for 
interpolation. Therefore^ degradation of resolution is 
suppressed in the diagonal directions and the L-shaped 
directions as well as in the vertical and horizontal directions. 
Embodiment: 7 . 

Next, a seventh embodiment corresponding to Claim 10 of 
the present invention will be described. 

The construction of this seventh embodiment is identical 
to those of the third to seventh embodiments except the 
correlation calculating method by the correlation calculation 
circuit 6. In the third to sixth embodiments, the correlation 
Vc in the vertical direction, the correlation He in the 
horizontal direction, the correlation Nr in the diagonal 
direction toward the lower right, the correlation Nl in the 
diagonal direction toward the lower left, the correlation Lul in 
the upper left L-shaped direction, the correlation Lur in the 
upper right L-shaped direction, the correlation Ldl in the lower 
left L-shaped direction, and the correlation Ldr in the lower 
right L-shaped direction are obtained by operation using the 
pixels of the same color. In this seventh embodiment, however, 
these correlations are obtained by operation using adjacent 
pixels of different colors, according to the following 
expressions . 

Vc= I Wu-Cyn I -h I Wd-Cyn | (38) 



Hc= I Wl-Cyn I + I Wr-Cyn | 



(39) 



Nr=| (Wu+Wl)/2-Cyn|+| ( Wd+Wr ) /2-Cyn | 
Nl=| (Wu+Wr)/2-Cyn|+| ( Wd+Wl ) /2-Cyn | 
Lul= I Wu-Cyn | + | Wl-Cyn | 



(42) 



(41) 



(40) 



Ldl= I Wd-Cyn | + | Wl-Cyn | 
Lur= I Wu-Cyn | + | Wr-Cyn | 



(44) 



(43) 



Ldr= I Wd-Cyn | + | Wr-Cyn | 



(45) 



Decision on the correlation direction and interpolation 
are carried out in like manner as those mentioned for the third 
to sixth embodiments. 

According to the seventh embodiment, since the 
correlations are obtained by operation between adjacent pixels 
of different colors, the precision in calculating the 
correlations is improved. 
Embodiment: 8 . 

Next, an eighth embodiment according to Claim 11 of the 
present invention will be described. 

The construction of this eighth embodiment is identical to 
those of the third to sixth embodiments except the interpolation 
method by the interpolation circuit 7. 

That is, in this eighth embodiment, when the target pixel 
Cyn obtained in any of the third to sixth embodjLments is 
subjected to interpolation, the luminance W' of this target 
pixel Cyn is obtained using only the peripheral W pixels without 
using the pixel Cyn itself, according to the following 



expressions . 

When the correlation direction is the vertical direction, 

W'=(Wu-i-Wd)/2 (46) 
When the correlation direction is the horizontal direction, 

W'=(Wl+Wr)/2 (47) 
When the correlation direction is the upper left L-shaped 
direction, 

W=(Wu+Wl)/2 (48) 
When the correlation direction is the lower left L-shaped 
direction, 

W'=(Wd+Wl)/2 (49) 
When the correlation direction is the upper right L-shaped 
direction, 

W»=(Wu+Wr)/2 (50) 
When the correlation direction is the lower right L-shaped 
direction, 

W=(Wd+Wr)/2 (51) 
When the correlation direction is other than those 
mentioned above, 

W ' = ( Wu+Wd+Wl+Wr ) /4 ( 52 ) 

According to this eighth embodiment, since the luminance 
signals are calculated using only the W pixels, the 
interpolation precision is improved, resulting in a high- 
resolution image without luminance variations. 
Embodiment 9 . 




Next^ a ninth embodiment corresponding to Claims 13 and 14 
of the present invention will be described with reference to 
figures 10 and 11. 

In this ninth embodiment^ when a target pixel to be 
interpolated is decided as having a correlation in a specific 
direction in the correlation detecting process according to any 
of the aforementioned embodiments, the color-difference signal 
(R-Y,B-Y) in the position of the target pixel is multiplied by a 
gain not larger than 1, regardless of the degree of the 
correlation. • 

Figure 10 shows the relationship between the degree of 
correlation in the direction which is decided as having the 
strongest correlation, and the gain by which the color- 
difference signal is to be multiplied. Although a pseudo 
chrominance signal tends to appear at the luminance edge, the 
pseudo chrominance can be suppressed when the interpolation 
circuit 7 has the function of multiplying the color-difference 
signal by a gain not larger than 1. Further, since the 
correlation detection circuit 6 may have the function of 
detecting the edge, the pseudo chrominance can be suppressed 
without providing a special luminance-edge detection circuit. 

Further, the gain by which the color-difference signal is 
multiplied may be varied according to the degree of correlation 
in the direction decided as having the strongest correlation. 
Figure 11 shows an example of relationship between the degree of 



correlation in the direction decided as having the strongest 
correlation and the gain by which the color-difference signal is 
multiplied. In this case, however, the stronger the correlation 
is, the smaller the degree of correlation is. 

In figure 11, a width Thl is given to the correlation, and 
the gain by which the color-difference signal is multiplied is 
gradually decreased for each width Thl. Generally, the larger 
the difference in luminance at the edge is, the thicker the 
pseudo chrominance becomes. That is, since the possibility of 
generating thick pseudo chrominance is higher as the correlation 
is smaller, the level of the color-difference signal can be 
reduced according to the possibility, whereby the pseudo 
chrominance can be suppressed effectively. 
Embodiment 10 . 

Hereinafter, a tenth embodiment corresponding to Claims 15 
and 16 of the present invention will be described with reference 
to figures 12, 13, 14, and 15. In these figures, the same 
reference numerals as those used for the aforementioned 
embodiments denote the same or corresponding parts. 

Figure 12 shows the structure of a solid-state color image 
pickup apparatus according to the tenth embodiment of the 
invention, and this is fundamentally identical to that shown in 
figure 6 except that a frequency characteristic adjustment 
circuit 10 is added to the structure of figure 6. The 
interpolation circuit 7 shown in figure 6 is separated into a 
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luminance signal interpolation circuit 8 and a color-difference 
signal interpolation circuit 9 in figure 12, and the frequency 
characteristic adjustment circuit 10 is inserted just before the 
color-difference signal interpolation circuit 9. The frequency 
characteristic adjustment circuit 10 adjusts the frequency of 
the image signal stored in the storage circuit 5, and outputs it 
to the color-difference signal interpolation circuit 9. 

Figure 13 is a schematic diagram for explaining the 
operation of the frequency characteristic adjustment circuit 10 
for restricting the frequency band to the low-pass band^ and 
adjusting the frequency characteristic of the color-difference 
signal. The frequency characteristic adjustment using a low- 
pass filter can be performed in the vertical direction, the 
horizontal direction, the vertical and horizontal directions, 
and the diagonal direction. When using the low-pass filter in 
the vertical direction, 2n+l (n=l,2,...) signals of the same 
color in the vertical direction with a target pixel (a pixel 
whose frequency characteristic is to be adjusted) in the center 
are used, and each signal is multiplied by a factor for deciding 
the filter characteristic, and the average of these signals is 
calculated. For example, in figure 13, when three signals of 
the same color in the vertical direction with the Cy23 as the 
target pixel are filtered by the low-pass filter, the output 
signal Cy23' from the frequency characteristic adjustment 
circuit 10 is represented* as follows. 



Cy23 ' = ( Cy03+Cy23+Cy43 ) /3 ( 53 ) 

Likewise, when using the low-pass filter in the horizontal 
direction, 2n+l (n=l,2,..,) signals of the same color in the 
horizontal direction with a target pixel (a pixel whose 
frequency characteristic is to be adjusted) in the center are 
used, and each signal is multiplied by a factor for deciding the 
filter characteristic, and the average of these signals is 
calculated. For example, in figure 13, when three signals of 
the same color in the horizontal direction with the Cy23 as the 
target pixel are filtered by the low-pass filter with all of the 
factors being 1, the output signal Cy23 ' from the frequency 
characteristic adjustment circuit 10 is represented as follows. 

Cy2 3 ' = ( Cy2 1+Cy23+Cy25 ) /3 ( 54 ) 

Likewise, when using the low-pass filter in the vertical 
and horizontal directions, ( 2n+l ) X ( 2m+l ) ( n,m=l , 2 , . . . ) signals 
of the same color in the vertical and horizontal directions with 
a target pixel (a pixel whose frequency characteristic is to be 
adjusted) in the center are used, and each signal is multiplied 
by a factor for deciding the filter characteristic, and the 
average of these signals is calculated. For example, in figure 
13, when nine signals of the same color in the vertical and 
horizontal directions with the Cy23 as the target pixel are 
filtered by the low-pass filter with all of the factors being 1, 
the resultant signal is represented as follows. 

Cy23 '=(Cy01+Cy03+Cy05+Cy21+Cy23+Cy25+Cy41+Cy4 3+Cy45) /9 (55 ) 
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Likewise, when using the low-pass filter in the diagonal 
directions, 2n+2m+l ( n,m=l , 2 , . • . ) signals of the same color in 
the vertical and horizontal cross-shaped directions with a 
target pixel (a pixel whose frequency characteristic is to be 
adjusted) in the center are used, and each signal is multiplied 
by a factor for deciding the filter characteristic, and the 
average of these signals is calculated. For example, in figure 
13, when five signals of the same color in the vertical and 
horizontal cross-shaped directions with the Cy23 as the target 
pixel are filtered by the low-pass filter with all of the 
factors being 1, the resultant signal is represented as follows. 



The above-described operation for adjusting the frequency 
characteristic is performed on all of the chrominance signals 
which are required for compositing the color-difference signal 
by interpolation. 

For example, when compositing the color-difference signal 
using the Cy23 as the target pixel and the W22 and W24 as the 
peripheral pixels, in order to apply the low-pass filter in the 
vertical and horizontal directions, the chrominance signals 
required for interpolation are calculated as shown in the 
following expressions (57), (58), and (59). 

W22 ' = (W00+W02+W04+W20+W22+W24+W40+W42+W44 ) /9 ( 57 ) 

Cy2 3 ' = (Cy01+Cy03+Cy05+Cy21+Cy23+Cy25+Cy4H-Cy43+Cy45 ) /9 (58 ) 

W24 ' = (W02+W04+W06+W20+W22+W24+W26+W42+W44+W46 ) /9 (59 ) 



Cy23 '=(Cy03+Cy21+Cy23+Cy25+Cy43 ) /9 
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In the color-difference signal interpolation circuit 9 
shown in figure 12 , the R-Y color-difference signal is 
calculated using the Cy23*, W22 • ^ and W24* according to the 
following expression. 

R-Y=AX (W22 '+W24 ' )-BXCy23 ' (60) 
wherein A and B are constants depending on the white balance or 
the like. Further, when the Ye pixels are subjected to the 
above-described frequency characteristic adjustment and color- 
difference signal compensation using the same positional 
relationship as described above, the B-Y color-difference signal 
can be obtained. 

Figure 14 shows the amplitude characteristics 11 using 
three-point averaging as a method of frequency characteristic 
adjustment for the chrominance signal, and the amplitude 
characteristics 12 using linear interpolation as a method of 
interpolation for the color-difference signal, and these 
characteristics combined are shown by the solid line. The 
abscissa shows the frequency, and the sampling frequency of each 
chrominance signal is indicated by tT . As shown in figure 14, 
when linear interpolation is performed using the chrominance 
signal subjected to frequency characteristic adjustment, the 
frequency component including aliasing, which appears in the 
vicinity of TV /2 of the chrominance signal (shown by the dashed 
line in figure 15), is reduced. 

In this tenth embodiment, when the chrominance signal 



stored in the storage circuit 5 contains high frequency 
components, the frequency characteristic adjustment circuit 10 
reduces the frequency component including aliasing, and the 
color-difference signal interpolation circuit 9 composites the 
color-difference signal by interpolation using the chrominance 
signal whose frequency characteristic is adjusted, whereby the 
pseudo chrominance signal is reduced. 
Embodiment 1 1 * 

Hereinafter, an eleventh embodiment corresponding to 
Claims 17 and 18 of the present invention will be described with 
reference to figure 16. In figure 16, the same reference 
numerals as those used for the aforementioned embodiments denote 
the same constituents. 

Figure 16 shows the structure of a solid-state color image 
pickup apparatus according to the eleventh embodiment, and it is 
fundamentally identical to the structure shown in figure 12 
except that the frequency characteristic adjustment circuit 10 
is controlled by the output from the correlation detection 
circuit 6. 

In this construction, when the correlation detection 
circuit 6 decides that there is a correlation direction, the 
frequency characteristic of the chrominance signal of a target 
pixel to be interpolated is adjusted by the frequency 
characteristic adjustment circuit 10. Further, using the color 
signal of the target pixel whose frequency characteristic is 
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adjusted, the color-difference signal is calculated by the 
color-difference signal interpolation circuit 9. This process 
is identical to that mentioned for the tenth embodiment. 

On the other hand, when the correlation detection circuit 
6 decides that there is no correlation direction, the 
chrominance signal of the target pixel is sent to the color- 
difference signal interpolation circuit 9 without being 
processed in the frequency characteristic adjustment circuit 10. 
In the interpolation circuit 9, the color-difference signal is 
calculated using the chrominance signal and the luminance signal 
from the luminance signal interpolation circuit 8. 

As described above, in this eleventh embodiment, the 
frequency characteristic of the chrominance signal including 
high-frequency components and having the correlation direction 
is adjusted by the frequency characteristic adjustment circuit 
10, whereby occurrence of pseudo chrominance signals is reduced. 
On the other hand, since the chrominance signal having no 
correlation direction does not include pseudo chrominance 
components inherently, this chrominance signal need not be 
subjected to adjustment of frequency characteristic, and so the 
frequency components of the chrominance signal are not 
attenuated by adjustment of frequency characteristic, whereby 
the color reproducibility is maintained. 



APPLICABILITY IN INDUSTORY 
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As described above, in the solid-state color image pickup 
apparatus according to the present invention, four pieces of 
luminance data and two pieces of chrominance data are taken out 
of vertically and horizontally adjacent 4 pixels of the color 
separation filter placed on the surface of the solid-state image 
pickup device, whereby the luminance resolution is improved, and 
degradation of chrominance resolution is reduced. Therefore, 
this is valuable as a signal processing method of the solid- 
state color image pickup apparatus which obtains high resolution 
by performing interpolation between pixels. 



